Content utilization paramerization

ABSTRACT

Feedback regarding a presentation may be received from viewers who have viewed the presentation. In some instances, the presentation may be segmented into a plurality of segments and one or more discrete segments included in the plurality may be divided into a plurality of sub-segments. The presentation may be provided to the plurality of viewers and feedback regarding a discrete sub-segment of the plurality of sub-segments may be received from each of the plurality of viewers via the feedback mechanism. A report based on the received feedback information may then be generated.

RELATED APPLICATIONS

This application is a CONTINUATION of U.S. patent application Ser. No.15/610,270, filed May 31, 2017, which is a CONTINUATION of U.S. patentapplication Ser. No. 14/932,964, filed Nov. 4, 2015, now U.S. Pat. No.9,681,185, which is a NONPROVISIONAL of U.S. Provisional PatentApplication No. 62/075,113, filed Nov. 4, 2014, all of which are herebyincorporated by reference in their entireties.

FIELD OF INVENTION

The present invention relates to the field of receiving feedbackregarding a presentation from viewers who have viewed the presentation.

BACKGROUND

An area of ongoing research and development is in improving contentusefulness. Content usefulness can be measured from at least twoperspectives. From one perspective, a question can be posed regardinghow useful content is to the content consumer. For example, one couldconsider whether the content is what the content consumer needs to makea decision. From another perspective, a question can be posed regardinghow useful it is to a content owner when the content consumer consumesthe content. For example, one could consider whether the content causedthe content consumer to take an action that inured to the benefit of thecontent owner.

It is possible to detect actions surrounding content. For example,content can be ranked so as to show how much a content consumer “liked”the content. As another example, a system could detect whether an ad isclicked after a content consumer consumes content. Recognizing thatmaking content more useful and receiving rankings of content can improveexperiences of content consumers and/or content owners, it is desirableto improve the utility of content.

BRIEF DESCRIPTION OF THE DRAWINGS

The present application is illustrated by way of example, and notlimitation, in the figures of the accompanying drawings, in which:

FIG. 1 depicts a diagram of an example of a system.

FIG. 2 depicts a diagram of an example of a content segmentation server.

FIG. 3 depicts a diagram of an example of a segmented content datastructure.

FIG. 4 depicts a diagram of an example of a segmented contentutilization parameterization system.

FIG. 5 depicts a flowchart of an example of a method for segmentedcontent utilization.

FIGS. 6, 7A-7B and 8 show reaction graphs.

WRITTEN DESCRIPTION

FIG. 1 depicts a diagram 100 of an example of a real-time response tocontent measuring system. The diagram 100 includes a computer-readablemedium 102, a content server 104 coupled to the computer-readable medium102, a content segmentation server 106 coupled to the computer readablemedium 102, and a segmented content utilization parameterization server108 coupled to the computer-readable medium 102.

In the example of FIG. 1, the computer-readable medium 102 can include anetworked system that includes several computer systems coupledtogether, such as the Internet, or a device for coupling components of asingle computer, such as a bus. The term “Internet” as used hereinrefers to a network of networks that uses certain protocols, such as theTCP/IP protocol, and possibly other protocols such as the hypertexttransfer protocol (HTTP) for hypertext markup language (HTML) documentsthat make up the World Wide Web (the web).

Content is often provided by content servers, which are referred to asbeing “on” the Internet. A web server, which is one type of contentserver, is typically at least one computer system which operates as aserver computer system and is configured to operate with the protocolsof the web and is coupled to the Internet. The physical connections ofthe Internet and the protocols and communication procedures of theInternet and the web are well known to those of skill in the relevantart. For illustrative purposes, it is assumed the computer-readablemedium 102 broadly includes, as understood from relevant context,anything from a minimalist coupling of the components illustrated in theexample of FIG. 1, to every component of the Internet and networkscoupled to the Internet.

A computer system, as used in this paper, is intended to be construedbroadly. In general, a computer system will include a processor, memory,non-volatile storage, and an interface. A typical computer system willusually include at least a processor, memory, and a device (e.g., a bus)coupling the memory to the processor.

The processor can be, for example, a general-purpose central processingunit (CPU), such as a microprocessor, or a special-purpose processor,such as a microcontroller.

The memory can include, by way of example but not limitation, randomaccess memory (RAM), such as dynamic RAM (DRAM) and static RAM (SRAM).The memory can be local, remote, or distributed. The term“computer-readable storage medium” is intended to include physicalmedia, such as memory.

The bus can also couple the processor to the non-volatile storage. Thenonvolatile storage is often a magnetic floppy or hard disk, amagnetic-optical disk, an optical disk, a read-only memory (ROM), suchas a CD-ROM, EPROM, or EEPROM, a magnetic or optical card, or anotherform of storage for large amounts of data. Some of this data is oftenwritten, by a direct memory access process, into memory during executionof software on the computer system. The non-volatile storage can belocal, remote, or distributed. The non-volatile storage is optionalbecause systems can be created with all applicable data available inmemory.

Software is typically stored in the non-volatile storage. Indeed, forlarge programs, it may not even be possible to store the entire programin the memory. Nevertheless, it should be understood that for softwareto run, if necessary, it is moved to a computer-readable locationappropriate for processing, and for illustrative purposes, that locationis referred to as the memory in this paper. Even when software is movedto the memory for execution, the processor will typically make use ofhardware registers to store values associated with the software, andlocal cache that, ideally, serves to speed up execution. As used herein,a software program is assumed to be stored at any known or convenientlocation (from non-volatile storage to hardware registers) when thesoftware program is referred to as “implemented in a computer-readablestorage medium.” A processor is considered to be “configured to executea program” when at least one value associated with the program is storedin a register readable by the processor.

The bus can also couple the processor to the interface. The interfacecan include one or more of a modem or network interface. It will beappreciated that a modem or network interface can be considered to bepart of the computer system. The interface can include an analog modem,isdn modem, cable modem, token ring interface, satellite transmissioninterface (e.g. “direct PC”), or other interfaces for coupling acomputer system to other computer systems. The interface can include oneor more input and/or output (I/O) devices. The I/O devices can include,by way of example but not limitation, a keyboard, a mouse or otherpointing device, disk drives, printers, a scanner, and other I/Odevices, including a display device. The display device can include, byway of example but not limitation, a cathode ray tube (CRT), liquidcrystal display (LCD), or some other applicable known or convenientdisplay device.

In one example of operation, the computer system can be controlled byoperating system software that includes a file management system, suchas a disk operating system. File management systems are typically storedin non-volatile storage and cause the processor to execute the variousacts required by the operating system to input and output data and tostore data in the memory, including storing files on the non-volatilestorage. One example of operating system software with associated filemanagement system software is the family of operating systems known asWindows® from Microsoft Corporation of Redmond, Wash., and theirassociated file management systems. Another example of operating systemsoftware with its associated file management system software is theLinux operating system and its associated file management system.Another example of operating system software with associated filemanagement system software is VM (or VM/CMS), which refers to a familyof IBM virtual machine operating systems used on IBM mainframesSystem/370, System/390, zSeries, System z, and compatible systems,including the Hercules emulator for personal computers.

Some portions of this paper may be presented in terms of algorithms andsymbolic representations of operations on data bits within a computermemory. These algorithmic descriptions and representations are the meansused by those skilled in the data processing arts to most effectivelyconvey the substance of their work to others skilled in the art. Analgorithm is here, and generally, conceived to be a self-consistentsequence of operations leading to a desired result. The operations arethose requiring physical manipulations of physical quantities. Usually,though not necessarily, these quantities take the form of electrical ormagnetic signals capable of being stored, transferred, combined,compared, and otherwise manipulated. It has proven convenient at times,principally for reasons of common usage, to refer to these signals asbits, values, elements, symbols, characters, terms, numbers, or thelike.

It should be borne in mind, however, that all of these and similar termsare to be associated with the appropriate physical quantities and aremerely convenient labels applied to these quantities. Unlessspecifically stated otherwise as apparent from the following discussion,it is appreciated that throughout the description, discussions utilizingterms such as “processing” or “computing” or “calculating” or“determining” or “displaying” or the like, refer to the action andprocesses of a computer system, or similar electronic computing device,that manipulates and transforms data represented as physical(electronic) quantities within the computer system's registers andmemories into other data similarly represented as physical quantitieswithin the computer system memories or registers or other suchinformation storage, transmission or display devices.

The algorithms and displays presented herein are not necessarilyinherently related to any particular computer or other apparatus.Various general purpose systems may be used with programs to configurethe general purpose systems in a specific manner in accordance with theteachings herein, or it may prove convenient to construct specializedapparatus to perform the methods of some embodiments. The requiredstructure for a variety of these systems will appear from thedescription below. In addition, the techniques are not described withreference to any particular programming language, and variousembodiments may thus be implemented using a variety of programminglanguages. Referring once again to the example of FIG. 1, the contentserver 104 is coupled to the computer-readable medium 102. The contentserver 104 can be implemented on a known or convenient computer system.Only one content server is illustrated in FIG. 1, but it should beunderstood that specific implementations could have multiple contentservers. Moreover, partial functionality might be provided by a firstdevice and partial functionality might be provided by a second device,where together the first and second devices provide the fullfunctionality attributed to the content server 104.

As used in this paper, a server is a physical computer (a computerhardware system) dedicated to run one or more services (as a host) toserve the needs of other computers on a network. Depending on thecomputing service that it offers it could be a database server, fileserver, mail server, print server, web server, gaming server, or someother kind of server. In the context of client-server architecture, asused in this paper, a server includes an engine that serves the requestsof other engines, the “clients;” the “server” engine performs somecomputational task on behalf of “client” engines. The clients areconnected to the server via the computer readable medium 102, either onthe same computer or through a network.

An engine, as used in this paper, includes a dedicated or sharedprocessor and, typically, firmware or software modules that are executedby the processor. Depending upon implementation specific or otherconsiderations, an engine can be centralized or its functionalitydistributed. An engine can include special purpose hardware, firmware,or software embodied in a computer-readable medium for execution by theprocessor.

In the example of FIG. 1, the content server 104 includes a contentcreation engine 110 and a content datastore 112. As its name suggests,the content creation engine 110 creates content. The content can becreated via multiple different operations (e.g., in parallel) andcompiled, generated and modified in successive operations (e.g., inseries), or some combination thereof. Depending upon context, contentmay refer to the content of a medium divorced from the medium itself.“Media” and some compound words that include “media” (e.g. multimedia,hypermedia) often refer to a type of content. The distinction betweenmedium and content is less clear when referring to interactive elementsthat contain information and are then contained in interactive media,such as GUI widgets contained in software. Due to the difficulty informing a clear distinction in all cases between content and media, theterm content is intended to mean either or both, depending upon thecontext in which the word is used. Thus, content can refer to theunderlying content, layout, format, media, or a subset of these. Thecontent creation engine 110 stores the content in the content datastore112.

The content datastore 112 and other datastores described in this paper,can be implemented, for example, as software embodied in a physicalcomputer-readable medium on a general- or specific purpose machine, infirmware, in hardware, in a combination thereof, or in an applicableknown or convenient device or system. Datastores described in this paperare intended, if applicable, to include any organization of data,including tables, comma-separated values (CSV) files, traditionaldatabases (e.g., SQL), or other known or convenient organizationalformats.

In an example of a system where the content datastore 112 is implementedas a database, a database management system (DBMS) can be used to managethe content datastore 112. In such a case, the DBMS may be thought of aspart of the content datastore 112 or as part of the content server 104,or as a separate functional unit (not shown). A DBMS is typicallyimplemented as an engine that controls organization, storage,management, and retrieval of data in a database. DBMSs frequentlyprovide the ability to query, backup and replicate, enforce rules,provide security, do computation, perform change and access logging, andautomate optimization. Examples of DBMSs include Alpha Five, DataEase,Oracle database, IBM DB2, Adaptive Server Enterprise, FileMaker,Firebird, Ingres, Informix, Mark Logic, Microsoft Access, InterSystemsCache, Microsoft SQL Server, Microsoft Visual FoxPro, MonetDB, My SQL,PostgreSQL, Progress, SQLite, Teradata, CSQL, OpenLink Virtuoso,Daffodil DB, and OpenOffice.org Base, to name several.

Database servers can store databases, as well as the DBMS and relatedengines. Any of the datastores described in this paper could presumablybe implemented as database servers. It should be noted that there aretwo logical views of data in a database, the logical (external) view andthe physical (internal) view. In this paper, the logical view isgenerally assumed to be data found in a report, while the physical viewis the data stored in a physical storage medium and available to aspecifically programmed processor. With most DBMS implementations, thereis one physical view and an almost unlimited number of logical views forthe same data.

A DBMS typically includes a modeling language, data structure, databasequery language, and transaction mechanism. The modeling language is usedto define the schema of each database in the DBMS, according to thedatabase model, which may include a hierarchical model, network model,relational model, object model, or some other applicable known orconvenient organization. An optimal structure may vary depending uponapplication requirements (e.g., speed, reliability, maintainability,scalability, and cost). One of the more common models in use today isthe ad hoc model embedded in SQL. Data structures can include fields,records, files, objects, and any other applicable known or convenientstructures for storing data. A database query language can enable usersto query databases, and can include report writers and securitymechanisms to prevent unauthorized access. A database transactionmechanism ideally ensures data integrity, even during concurrent useraccesses, with fault tolerance. DBMSs can also include a metadatarepository; metadata is data that describes other data.

Referring once again to the example of FIG. 1, the content segmentationserver 106 is coupled to the computer-readable medium 102. The contentsegmentation server 106 can be implemented on a known or convenientcomputer system. Only one content segmentation server is illustrated inFIG. 1, but it should be understood that specific implementations couldhave multiple content servers. Moreover, partial functionality might beprovided by a first device and partial functionality might be providedby a second device, where together the first and second devices providethe full functionality attributed to the content segmentation server106. Conceptually, logically, or physically, the content segmentationserver 106 and the content server 104 can be combined to form a contentcreation and segmentation server that facilitates the creation ofsegmented content.

In the example of FIG. 1, the content segmentation server 106 includes acontent segmentation engine 114 and a segmented content datastore 116.The content segmentation engine 114 can segment content from the contentdatastore 112 using rules and/or input from a user. Segmentation isdescribed later in greater detail. The results of the segmentationprocess are stored as segmented content in the segmented contentdatastore 116.

Referring once again to the example of FIG. 1, the segmented contentutilization parameterization server 108 is coupled to thecomputer-readable medium 102. The segmented content utilizationparameterization server 108 can be implemented on a known or convenientcomputer system. Only one segmented content utilization parameterizationserver is illustrated in FIG. 1, but it should be understood thatspecific implementations could have multiple servers. Moreover, partialfunctionality might be provided by a first device and partialfunctionality might be provided by a second device, where together thefirst and second devices provide the full functionality attributed tothe segmented content utilization parameterization server 108.Conceptually, logically, or physically, the segmented contentutilization parameterization server 108 and the content segmentationserver 106 can be combined to form a content segmentation andutilization parameterization server that facilitates segmenting contentand obtaining utilization parameters therefor.

In the example of FIG. 1, the segmented content utilizationparameterization server 108 includes a segmented content utilizationparameterization engine 118 and a content utilization datastore 120. Thesegmented content utilization parameterization engine 118 can obtainutilization feedback associated with consumption of the segmentedcontent by content consumers for providing to content presenters tofacilitate a better understanding of or information useful fordetermining how to change the content. Mechanisms for obtaining thefeedback and providing useful information are described later in greaterdetail. The useful information is stored in the content utilizationdatastore 120.

FIG. 2 depicts a diagram 200 of an example of a content segmentationserver. The diagram 200 includes a content segmentation engine 202, auser interface 204 coupled to the content segmentation engine 202, acontent datastore 206 coupled to the content segmentation engine 202,and a segmented content datastore 208 coupled to the contentsegmentation engine 202.

In the example of FIG. 2, in operation, the content segmentation engine202 receives instructions from a user over the user interface 204. Thecontent segmentation engine 204 can also display content or otherinformation to the user over the user interface 204. The user interface204 can include applicable technology, such as a network interface,browser, drivers, and other components that enable a user to utilize thecontent segmentation engine 204. Depending upon the implementation, theuser interface 204 could be considered optional because the contentsegmentation engine 202 could operate using stored rules. However, it isexpected that users will often wish to provide input regarding thesegmentation process, particularly after received feedback, if any, fromconsumers of the segmented content.

In the example of FIG. 2, in operation, the content datastore 206includes content that can be converted into segmented content. Thecontent can be of an applicable content type that can depend onimplementation-specific factors. Depending upon the implementation, thecontent can be configured for use in a specific medium (e.g.,multimedia, POWERPOINT® slide, etc.). If the content is stored in a“generic” format, the content segmentation engine 202 will likelyinclude a conversion engine for converting from the generic format to astandard, proprietary, or other format. It may be desirable in any casefor the content segmentation engine 202 to have some conversioncapabilities, regardless of the format of the stored content, becausethat reduces media-specific limitations.

In the example of FIG. 2, in operation, the segmented content datastore208 includes content that has identifiable segments. Segmented contenthas a specific data structure that comprises multiple content segmentswith segment identifiers. The segment identifiers facilitate tracking ofutilization with respect to the specific segments. In a specificimplementation, the segmented content includes multiple content segmentsof video content, such as an mpeg. In this specific implementation, thempeg will have a duration that can be measured, but the mpeg is notbroken into discrete time units. During transmission, the mpeg is alsotypically broken into packets (also referred to using other terminology,such as “datagram” depending upon the transmission technology, OSIlayer, or other factors), but the mpeg will not typically be broken intopackets until the relevant stimulus (e.g., transmission over a wirelesslink) come into play. Thus, the mpeg is treated as a discrete unit ofvariable size. The segmented content includes additional information,such as metadata that identifies segments of the mpeg that correspond tospecific segments. The metadata can make use of the duration of thempeg, or more specifically the timeline, to identify relevant segments.In a specific implementation, the segments are of variable size.

FIG. 3 depicts a diagram 300 of an example of a segmented content datastructure. The diagram 300 includes a segmented content substructure 302and a metadata substructure 304. In the example of FIG. 3, the segmentedcontent substructure 302 includes content segments 306-1 to 306-N(referred to collectively as “content segments 306”). The differentsizes of the content segments 306 in the example of FIG. 3 is intendedto illustrate that the content segments can be of non-uniform size. Theactual size of the content segments 306 can be affected by a userproviding input regarding the segmentation process, rules used by acontent segmentation engine, or other factors. The size of the contentsegments 306 may or may not be limited to a predefined range. In animplementation that does not include a predefined range, the smallestsize can be limited by practicality or the smallest meaningful unit ofmeasure available and the largest size can be bound by the size of thecontent. In a specific implementation, the segmented contentsubstructure 302 includes a data structure that is unsegmented, and themetadata of the metadata substructure 304 can be used to identify thesegmentation. (In this specific implementation, converting content tosegmented content can be accomplished without actually modifying thecontent.)

In the example of FIG. 3, the metadata substructure 304 includes segmentidentifiers (IDs) 308-1 to 308-N (referred to collectively as “segmentIDs 308”). In the example of FIG. 3, the segment IDs 308 correspond tothe beginning of respective ones of the content segments 306. In analternative, the segment IDs 308 could instead or in addition correspondto the ends of respective ones of the content segments 306. The metadatasubstructure 304 can include other metadata, as well, but a minimalistmetadata substructure 304 would likely include only the segment IDs 308and media- or technologyspecific metadata (standards-based orproprietary), if any.

Referring once again to the example of FIG. 2, in operation, the contentsegmentation engine 202 converts content in the content datastore 206into segmented content that is stored in the segmented content datastore208. The datastores need not be physically distinct, and content that ispartially segmented or metadata about the ultimate content segmentation,can be considered part of either datastore, but is treated as part ofthe segmented content datastore 208 for illustrative purposes; data thatis stored in association with the segmentation of content is treated aspart of the segmented content datastore 208 in this paper.

FIG. 4 depicts a diagram 400 of an example of a segmented contentutilization parameterization system. The diagram 400 includes a computerreadable medium 402, a segmented content utilization parameterizationserver 404 coupled to the computer readable medium 402, segmentedcontent consumer clients 406 coupled to the computer readable medium402, and an interested party device 408 coupled to the computer readablemedium 402. In the example of FIG. 4, the computer readable medium 402is of a size that varies by implementation, but is likely to be betweenLAN-sized and WAN sized (e.g., the Internet and/or a public switchedtelephone network (PSTN)).

In the example of FIG. 4, the segmented content utilizationparameterization server 404 includes a focus group identification engine410, a focus group datastore 412, a segmented content presentationengine 414, a segmented content datastore 416, a segmented contentutilization parameterization engine 418, a content utilization datastore420, and a segmented content utilization reporting engine 422.

The focus group identification engine 410 can use the focus groupdatastore 412 to identify a first focus group for first segmentedcontent. Alternatively or in addition, the focus group identificationengine 410 can invite an identifiable group of users to participate. Theinvitation can be sent via a channel different from the channel by whichthe segmented content is ultimately presented (e.g., the invitationcould be sent via an email service and the segmented content could bepresented in a web page). The term “focus group” is not intended tolimit participation; the term is intended to be applied broadly. Forexample, in the context of human resources, the “focus group” could bepotential employers who receive segmented content associated with acandidate. As other examples, the focus group could include potentialinvestors in businesses associated with segmented content, potentialpurchasers of products associated with segmented content, potentialsocial matches with people associated with segmented content, to nameseveral. An invitation can include additional materials, such as aprompt to fill out a profile necessary for participation, a prompt tosign a non-disclosure agreement (NDA) or other contract, to name acouple. The contents of the invitation and the requirements forparticipation, if any, can depend upon the implementation and/orconfiguration (e.g., as required by the content creator, segmentedcontent presenter, etc.). In a specific implementation, the invitationis provided in the form of a quick response (QR) code, which can act asa guide to a display of relevant segmented content. The QR code can beprinted on a business card or other physical material. The invitationcould also be provided using a shortrange technology to a nearbyperson's mobile device. Advantageously, the focus group identificationengine 410 can facilitate crowdsourcing of funding, investment,recruiting, market validation, and other activities.

The segmented content presentation engine 414 can provide firstsegmented content in the segmented content engine 416 to a first focusgroup identified by the focus group identification engine 410. The firstsegmented content can be presented on demand to any member of the firstfocus group, within constraints that vary by implementation and/orconfiguration (e.g., within a predetermined time period for allparticipants, within a period of time from receipt of the invitation,until the first segmented content is outdated (potentially includingreplacing the first segmented content with new content), depending uponthe parameters of the invitation, depending upon the state of completionof a participant's profile, etc.). Alternatively, the first segmentedcontent can be presented at a specific time, requiring participants to“tune in” at the appropriate time.

In a specific implementation, the segmented content presentation engine414 presents the first segmented content with a feedback mechanism. Thefeedback mechanism can have an appearance 22 that varies depending uponthe implementation, but in any case provides content utilizationfeedback that can be applied to a specific segment of the firstsegmented content. For example, a slider can be presented with a video.Content consumers can adjust the slider depending upon how much theylike the content at any given time. Meaning of the word “like” can varydepending upon the context (e.g., in an educational context, “like” canmean the content consumer feels like they are learning something). In aspecific implementation, the feedback mechanism includes sensors todetect facial expressions (e.g., using facial recognition technology),verbal cues (e.g., using voice analysis), physical treatment of a mobiledevice (e.g., pressure, movement, etc.). The segmented contentutilization parameterization engine 418 receives data associated withutilization of the first segmented content for storage in the contentutilization datastore 420. Content utilization parameterization caninclude consideration of feedback associated with the feedbackmechanism. The feedback can be explicitly identified in association witha segment or the segmented content utilization parameterization engine418 can use data contained within feedback to identify the relevantsegment to which the feedback should be associated (e.g., a ratingassociated with a particular time in a video clip can be applied to asegment that is known to last from before the particular time to afterthe particular time). Because the feedback mechanism is timevarying, itmay be desirable to correct for time it takes to process new data (e.g.,consider a first segment 23 associated with a high rating and a secondsegment immediately following the first segment associated with a lowrating; for a period of time after the first segment ends, the feedbackcould be “high” until a content consumer decides the second segment isnot as good, whatever that means in the given context).

The segmented content utilization reporting engine 422 creates reportsfrom data in the content utilization datastore 420. The reports areintended to give an interested third party information about contentutilization. Because utilization data can be sufficiently discrete toprovide information about specific segments of content, the reports canenable the interested third party to decide whether to change therelevant segments in a manner responsive to the data. For example, aninterested third party might be motivated to change a segment to make itmore appealing to a specific demographic determinable from a focus groupproviding feedback about the content. The interested party, as used inthis paper, is a person or legal entity with an interest inunderstanding how appealing first segmented content is to a subset ofconsumers of the content. The interested party may or may not be acontent creator and/or a segmented content creator.

In the example of FIG. 4, the segmented content consumer clients 406include content consumer devices 424-1 to 424-N (collectively referredto as the content consumer devices 424). In a specific implementation,the content consumer devices 424 are used to play first segmentedcontent 24 and input feedback associated with the first segmentedcontent in a manner that enables the feedback to be discretelyassociated with specific ones of the segments of the first segmentedcontent. A content consumer may or may not receive an invitation, ifany, on the same device that is used to play the first segmentedcontent. (E.g., the invitation could be received on a first device andthe segmented content played on a different device.) Because the device“plays” the first segmented content, the device can be referred to as a“playback device.” Some content (or the media in which the content ispresented) may have a different verb associated with its playback. Forexample, still pictures, such as jpegs, might be referred to as “viewed”instead of played. Slides might be referred to as “showed” instead ofplayed. For convenience, the verb “to play” is intend to cover the useof any of these verbs in this paper.

In a specific implementation, content consumer devices 424 do not playthe first segmented content. Rather, the first segmented content isplayed by another device and the content consumer devices 424 are usedfor the sole purpose of providing feedback that can be discretelyassociated with specific ones of the segments of the first segmentedcontent. For example, the first segmented content could be presented ona screen that is visible to an audience. One or more of the audiencemembers can use one or more of the content consumer devices 424 toprovide feedback regarding the first segmented content that is playedfor the audience. In an alternative, the first segmented content can 25be a live-action (or otherwise unsegmented) presentation that issegmented after the fact. (In this alternative, the system of theexample of FIG. 4 could include a presented content segmentation engine,not shown.)

In the example of FIG. 4, the interested party device 408 can receivesegmented content utilization reports from the segmented contentutilization reporting engine 422. An associated interested party can usethe interested party device 408 (or some other device) to modify one ormore segments of the segmented content that was played by or for theapplicable focus group. Such modification can be made by a contentsegmentation engine (see, e.g., FIG. 2) or a segmented content editingengine (not shown).

FIG. 5 depicts a flowchart 500 of an example of a method for segmentedcontent utilization. The flowchart 500 is presented as a series ofdiscrete modules. Alternative flowcharts can change the order of themodules or reorder the modules for parallel execution, if applicable. Inthe example of FIG. 5, the flowchart 500 starts at module 502 withcreating first segmented content. Segmented content can be created by,at a minimum, defining a starting and ending point for each of aplurality of segments of content. Identifying a starting point for eachsegment is generally sufficient to implicitly define a starting andending point for each of the plurality of segments of content.Similarly, identifying an ending point for each segment is generallysufficient to implicitly define a starting and ending point 26 for eachof the plurality of segments of content. In an implementation thatincludes “dead space” that is not treated as a segment, it may bedesirable to explicitly define both starting and ending points for eachrelevant segment. Because the start and end of content is often known,it may be desirable to identify breaks between segments, as opposed tostarting points for each segment (because the starting point of thefirst segment in an implementation that includes no dead space will bethe start of the content) or end points for each segment (because theending point of the last segment in an implementation that includes nodead space will be the end of the content), which will generally includeone less segment break data point than either start data points in animplementation that identifies the start of each segment or end datapoints in an implementation that identifies the end of each segment. Ina specific implementation, segmenting content involves creatingmetadata, as opposed to chunking segments of content into discrete,e.g., files.

In the example of FIG. 5, the flowchart 500 continues to module 504 withidentifying a focus group for the first segmented content. The focusgroup can be identified by identifying records or other data structuresin a datastore that match a certain criteria (e.g., accounts withprofiles that match a specific demographic). The focus group can beidentified by sending invitations to those on a focus group list thatmay or may not be formed by matching records in a datastore with acriteria. The focus 27 group can be identified over time using responsesfrom invitees. The responses may or may not include data that can bematched to specific criteria; when no such data is included, theresponse itself can be used as an identification of a focus groupparticipant. Depending on the implementation, the first segmentedcontent may or may not be identified or even identifiable when a focusgroup is identified or when invitations are sent. Of course, it will bedesirable to have the content both identified and available when aparticipant expects he or she should be able to play the content (e.g.,by clicking on a hyperlink).

In the example of FIG. 5, the flowchart 500 continues to module 506 withproviding the first segmented content to the focus groupcontemporaneously with a feedback mechanism. The feedback mechanism mayor may not be provided on a same device as the first segmented contentis played. For example, the feedback mechanism could be presented on amobile device of a person who is viewing the first segmented content ona movie screen. Feedback entered in accordance with the characteristicsof the feedback mechanism can be timestamped or otherwise timed to matchthe feedback to discrete segments of the first segmented content as itis presented. Feedback can be measured using an applicable sensor. In aspecific implementation, the feedback mechanism is a slider that can beadjusted during the presentation of the first segmented content toindicate greater or 28 lesser approval of the first segmented content. Atwo-dimensional slider (e.g., with a vertical and horizontal component)can be used to capture approval that consists of more than one metric(e.g., aesthetic and informational approval). While a desirable featureof the feedback mechanism is ease of use, it is theoretically possibleto include any practical number of sliders or input devices thatcorrespond to different aspects of approval.

In a specific implementation, the first segmented content is presentedone segment at a time with breaks between the segments. For example, afirst segment can play for a time and viewers of the first segment canbe prompted to provide feedback when the first segment ends. When thefeedback is received, a second segment can play for a time and viewersof the second segment can be prompted to provide feedback when thesecond segment ends. It may be desirable to present the first segmentedcontent in its entirety, without breaks for providing feedback, eitherbefore the content is presented in its discrete segments, or after thecontent is presented in discrete segments.

In the example of FIG. 5, the flowchart 500 continues to module 508 withreceiving feedback associated with a discrete segment of the firstsegmented content. The feedback mechanism may or may not include anawareness of the segments of the first segmented content. For example,the feedback mechanism can timestamp input and send the timestampedfeedback to a server for application to a discrete segment without thedevice on which the feedback mechanism is presented 29 ever receivingsegment IDs. When the first segmented content is presented in discretesegments, feedback can be applied to the just-presented segment.

In the example of FIG. 5, the flowchart 500 continues to module 510 withgenerating a report for an interested party including information aboutone or more discrete segments of the first segmented content. The reportenables the interested party to determine an impact of the firstsegmented content on the relevant focus group. To the extent demographicdata is known for participants in the focus group, the report can showimpact by demographic. In the example of FIG. 5, the flowchart 500continues to module 512 with modifying the first segmented content tocreate second segmented content. Advantageously, the interested partycan target specific discrete segments of the first segmented content forimprovement in response to the feedback that is associated with thespecific discrete segments. The second segmented content can then bepresented to the same or a different focus group as describedpreviously, or presented publicly or to groups matching a relevant(e.g., demographic) criteria if the interested party is satisfied withthe results. For example, a segmented video resume could be presented toa number of employers and improved using feedback from the employers,then presented to other employers. An individual who is looking for twojobs might have two different resumes, and only employers matching theappropriate criteria would see the relevant resume (e.g., a person couldhave an acting 30 CV and a computer programmer CV, and employers wouldbe presented with one or the other, but not both). It is possible to useboth a first segmented content and a second segmented content, wheresome of the segments are identical and some of the segments are moreattuned to consumers of the content with relevant criteria.

The present invention provides users with the ability to obtaininformation regarding a viewer's reaction to a video and/or audio/videopresentation for a plurality of viewers. In some instances the pluralityof viewer may be organized based on one or more characteristics (e.g.,age, gender, annual income, etc.).

In general a presentation is provided to a viewer and that viewerprovides feedback regarding the presentation at various points in thepresentation. A portion of the presentation played between the points inthe presentation may be referred to a segment as explained above. Viewerfeedback regarding a presentation, or a segment thereof, may be obtainedin any appropriate manner including, but not limited to a binary like ordon't like decision, a rating on a scale (e.g., 1-5 or 1-10), or abehavioral response (e.g., looking away from the screen, time spentlooking only the screen providing the presentation to the viewer,whether the viewer fast forwarded or rewound a segment of thepresentation, etc.).

Once the information is obtained, it may be analyzed and provided to auser and/or producer of the presentation in a number of ways as will bediscussed in detail below. Initially, an overview of the viewers'reactions to the presentation and/or a segment thereof may be presentedon, for example, an interactive dashboard provided by a display orcomputer monitor. The overview may provide information regarding, forexample, how many viewers watched the presentation, information aboutviewer characteristics, an overview of viewer reaction to thepresentation or a segment thereof, etc.

A user may be provided with various analytical and/or statisticalinformation regarding viewer response to the presentation in the formof, for example, a graph or table. For example, a user may be providedwith a window by which to view the 31 presentation at issue, informationregarding viewer reactions to the presentation, how many viewers viewedthe presentation, attention span of viewers when viewing thepresentation, etc.

The viewer information may be subjected to various filters and otherstatistical means of analyzing and/or filtering the information. Theviewer information may be presented to a user via a reaction graph ofpositive and negative feedback as a function of time, or segment. Itwill be understood by those of skill in the art that the intervals forthe time scale may be of any appropriate duration including, but notlimited to, seconds, minutes, hours, etc.

An example of a reaction graph 600 is provided in FIG. 6, whichillustrates like and dislike for a presentation as a function of time.In reaction graph 600, values above the 0 x-axis represent likes, valuesbelow the 0 x-axis represent dislikes and the white square and line thatconnects the white squares represent the average, or mean, number oflikes vs. dislikes.

Reaction graph 600 may also be used to determine the number ofreactions, or feedback instances, that occur for a particular segment ofthe presentation. For example, the largest number of feedback instancesoccur at 00.04 seconds and the smallest number of feedback instancesoccur at 00:24, 00:25, and 00:26 time. This information may be used todetermine viewer engagement with the presentation and how much attentionthey were paying to the presentation.

On some occasions, viewer information may include information mayinclude viewer comments about a segment and/or the presentation as awhole. In some embodiments, this information may be accessed by focusingin, or otherwise selecting a portion of reaction graph 600. In this way,a user may interact directly with the reaction graph 600 so as todetermine one or more factors that may contribute to a quantity of likeand/or dislikes.

In some embodiments, definition and/or selection of video segments maybe user configurable. A user may define a segment, or a plurality ofsegments, before, during, and/or after viewer information is gathered.For example, a user may define a segment according to duration (e.g.,segments defined by a time period like 5 seconds, 15 seconds, etc.),correlation to a storyline of the presentation (e.g., introduction,conclusion, 32 display of a logo, introduction of a concept, etc.),correlation to a different presentation, and so on.

In some embodiments, a user may define a segment while watching thepresentation and marking the presentation in some way. For example, auser may start and stop play of the presentation and define the portionof the presentation that occurs between the start and stop of play as asegment. Segments may be separate from one another on the timeline ormay over lap, either partially or totally on the timeline. A user mayalso adjust the portions of the presentation selected via stopping andstarting play the presentation by, for example, manually adjusting oneor more times that define the segment. To facilitate differentiationbetween segments, a user may be able to assign a name or otheridentifier to a defined segment.

In one embodiment, a user may define a segment by selecting a timeperiod from the timeline of a reaction graph, such as reaction graph600. For example, a user may want to select a portion of a reactiongraph that corresponds to a lot of likes as a first segment, a portionof the reaction graph that corresponds to a lot of dislikes as a secondsegment, and so on.

An example of user defined segmentation is provided in FIGS. 7A and 7B.FIG. 7A provides a page 700 that includes a second reaction graph 701 oflikes and dislikes for a presentation as a function of time in minutes701 and a timeline is segments 715. Reaction graph 701 depicts likes asvalues that extend upward from the 0 x-axis in one minute intervalsalong the time span of 00:00 to 00:30. Reaction graph 701 also depictsdislikes as values that extend downward from the 0 x-axis in one minuteintervals along the time span of 00:00 to 00:30. Reaction graph 701 alsodepicts a time span range of interest 710, or defined segment, withinthe presentation (in this case between 00:06 minutes and 00:17 minutes),which is depicted in FIG. 7B, wherein a zoom in, or magnified view ofthis time span of interest 710 is provided.

When the time span of interest 710 corresponds to a segment, a user mayaccess information regarding the segment by selecting The timeline ofuser defined segments 715 indicates segments 720A-720D of thepresentation that relate to different content elements of thepresentation. For example, segment 720A relates to a introductionsegment, segment 720B relates to a 33 dragging out segment, segment 720Crelates to a warning a friend segment, and segment 720D relates to alock car segment. In some instances, segments 720A-720D depicted on thetimeline of user defined segments 715 may correspond with a time withinthe presentation in which content related to these segments is providedto a viewer.

At times, a presentation and/or segment may be associated with one ormore viewer comments. A user may access these comments by, for example,selecting a segment from a reaction graph like reaction graphs 600and/or 700.

In some instances, a user may apply one or more filters to the viewerinformation used to make a reaction graph. Exemplary filters may applyto characteristics and/or demographics associated with a viewer (e.g.,age, gender, college attended, annual income, etc.), characteristics ofthe presentation, and when the presentation is viewed. FIG. 8 providesan exemplary filtered reaction graph 800 wherein there are three filtersapplied to the viewer information as shown in filter indicators 805A,805B, and 805C. More specifically, filter indicator 805A shows thatviewer information has been filtered according to the viewercharacteristics of a location being California, in the United States andof the male gender. Data filtered using filter indicator 805A isdepicted in reaction graph 800 as line 810A. Filter indicator 805B showsthat viewer information has been filtered according to association withthe college (SJSU, Berkeley). Data filtered using filter indicator 805Bis depicted in reaction graph 800 as line 810B. Filter indicator 805Cshows that viewer information has been filtered according to associationwith the male gender. Data filtered using filter indicator 805C isdepicted in reaction graph 800 as line 810C.

Providing reaction graph 800 enables a user to simultaneously viewviewer reactions as filtered according to different criteria. Thisenables a user to, for example, determine how the presentation is beingperceived by different types of viewers in order to, for example,determine whether a target audience is having a desired reaction to thepresentation and/or a segment thereof.

In another embodiment, a user may be able to compare two differentpresentations on, for example, one reaction graph or two differentreaction graphs that are placed side by side. Viewer information for twodifferent presentations also be provided to the user by table or chart.The viewer information for each of the two presentations 34 may bealigned with one another using, for example, a consistent time scale(i.e., the same intervals of time) and/or segment definition forpresenting the viewer information for each respective presentation.

And one embodiment, a user may gain your information by first uploadinga presentation to a website accessible by one or more viewers. The usermay then select one or more viewers to view the uploaded presentationaccording to, for example, demographic information and/or knowledge ofthe particular viewer. For example, a user may upload a presentation andthen invite particular individuals to view the presentation and providefeedback. On some occasions, a user may aggregate two or more viewerstogether into a panel of viewers.

A user may solicit that viewer information by, for example, sending andinvitation to an individual viewer, a plurality of individual viewers,and/or a panel of viewers. On some occasions, invitations may be set byemail, text (SMS message), or other medium (e.g., phone call). In someinstances, viewers may be selected using contacts of the user as wouldbe identified by for example, an address book or a social mediaplatform.

In some instances, viewer information may be used to evaluate a publicspeaking performance such as a product demonstration, a product/servicepitch to a group of investors, a commercial or advertisement, a movietrailer, and so on.

The following discussion relates to an exemplary process for gainingviewer information from a viewer. Initially, a viewer will be surveyedin order to ascertain, for example, viewer demographic information,location information, and awareness or familiarity with subject matter(e.g., brand, product category, individual making the presentation,etc.) associated with the presentation. The viewer may then watch thepresentation and provide feedback directly and/or indirectly. Examplesof directly provided feedback are viewer comments on the presentationand/or a segment thereof and an indication that the viewer likes ordislikes the presentation and/or a segment thereof. Examples ofindirectly provided feedback are viewer attention span as measured by,for example, a frequency of receiving feedback from the viewer and/oreye contact of the viewer on a screen providing the presentation.

On some occasions, the viewer feedback may be compared with the initial35 survey information to determine, for example, a change in viewerperception and/or a correlation between one or more demographiccharacteristics and the viewer feedback so as to, for example, determinea trend in perception of the presentation for various demographiccharacteristics when, for example, analyzing viewer information from aplurality of viewers.

In some instances, a viewer may be surveyed a period of time (e.g., afew hours, a day, a week, etc.) after viewing the presentation in orderto gauge, for example, whether the viewer remembers the presentationand/or has a lasting impression of the presentation. The results of thissurvey may be compared with those of the feedback provided while viewingthe presentation in order to, for example, determine any discrepanciestherebetween.

We claim:
 1. A method comprising: receiving a presentation; defining astarting and ending point for each of a plurality of segments of thepresentation and thereby creating a segmented presentation; defining astarting and ending point for each of a plurality of sub-segments of adiscrete segment of the plurality of segments; providing the segmentedpresentation, with the discrete segment that includes the plurality ofsub-segments, contemporaneously with the feedback mechanism to aplurality of viewers; receiving feedback information regarding adiscrete sub-segment of the plurality of sub-segments from each of theplurality of viewers via the feedback mechanism; and generating a reportbased on the received feedback information, wherein the report is basedon the received feedback information regarding the discrete sub-segment.2. The method of claim 1, wherein the discrete segment is a firstdiscrete segment, the method further comprising: receiving feedbackinformation regarding a second discrete segment of the plurality ofsegments included in the segmented presentation from each of theplurality of viewers, wherein the report is further based on thereceived feedback information regarding the second discrete segment. 3.The method of claim 1, further comprising: receiving a criterion fordefining the starting and ending point for each of the plurality ofsub-segments of the discrete segment from a user prior to the definingof the starting and ending point for each of the plurality ofsub-segments
 4. The method of claim 1, wherein the feedback informationreceived is either a positive indication or a negative indication, thereport comprising a graph that shows the number of positive indicationsand negative indications for the discrete segment as a function of time.5. The method of claim 1, further comprising: providing a plurality offilters by which a recipient of the report can organize informationincluded in the report to a user; receiving a selection of one or morefilters from the recipient; and filtering the received feedbackinformation using the selected one or more filters, wherein the reportis further generated using the filtered feedback.
 6. The method of claim1, further comprising: receiving one or more criteria for the pluralityof viewers from a user, wherein the segmented presentation is onlyprovided to viewers that match the received one or more criteria.
 7. Themethod of claim 1, wherein the received feedback information includes aviewer comment regarding the discrete segment, the viewer comment beingreceived from at least one of the plurality of viewers via the feedbackmechanism.
 8. The method of claim 1, wherein the feedback informationupon which the report is based includes a viewer comment regarding thediscrete segment, the viewer comment being received from at least one ofthe plurality of viewers via the feedback mechanism.
 9. A methodcomprising: receiving a presentation; defining a starting and endingpoint for each of a plurality of segments of the presentation andthereby creating a segmented presentation; receiving one or morecriteria for the plurality of viewers from a user; providing thesegmented presentation contemporaneously with a feedback mechanism to aplurality of viewers that match the received one or more criteria;receiving feedback information regarding the presentation from each ofthe plurality of viewers via the feedback mechanism; and generating areport for a user based on the received feedback information.
 10. Themethod of claim 9, wherein the feedback information received is either apositive indication or a negative indication, the report comprising agraph that shows the number of positive indications and negativeindications for the discrete segment as a function of time.
 11. Themethod of claim 9, further comprising: providing a plurality of filtersby which a recipient of the report can organize information included inthe report to a user; receiving a selection of one or more filters fromthe recipient; and filtering the received feedback information using theselected one or more filters, wherein the report is further generatedusing the filtered feedback.
 12. The method of claim 9, furthercomprising: receiving one or more criteria for the plurality of viewersfrom a user, wherein the segmented presentation is only provided toviewers that match the received one or more criteria.
 13. The method ofclaim 9, wherein the received feedback information includes a viewercomment regarding the presentation, the viewer comment being receivedfrom at least one of the plurality of viewers via the feedbackmechanism.
 14. The method of claim 9, wherein the feedback informationupon which the report is based includes a viewer comment regarding thepresentation, the viewer comment being received from at least one of theplurality of viewers via the feedback mechanism.